Siamese Network with Interactive Transformer for Video Object Segmentation

Semi-supervised video object segmentation (VOS) refers to segmenting the target in remaining frames given its annotation first frame, which has been actively studied recent years. The key challenge lies finding effective ways exploit spatio-temporal context of past help learn discriminative representation current frame. In this paper, we propose a novel Siamese network with specifically designed interactive transformer, called SITVOS, enable propagation from historical frames. Technically, use transformer encoder and decoder handle frame separately, i.e., encodes robust frames, while takes feature embedding as query retrieve output. To further enhance representation, interaction module (FIM) is devised promote information flow between decoder. Moreover, employ architecture extract backbone features both enables reuse more efficient than existing methods. Experimental results on three challenging benchmarks validate superiority SITVOS over state-of-the-art Code available at https://github.com/LANMNG/SITVOS.